An image sensing system

ABSTRACT

According to an aspect, there is provided an image sensing system comprising: a rolling shutter image sensor comprising an array of pixels in a frame arranged into a plurality of image lines extending along a width direction, and distributed in a scanning direction which is perpendicular to the width direction, wherein the rolling shutter image sensor is configured to scan the frame by sequentially scanning each image line along the scanning direction, wherein scanning each image line includes making the pixels in each image line sensitive to light for a predetermined exposure time, and then determining an intensity readout for each pixel of the respective image line; a lens system configured to project at least two similar outgoing images of an object onto the image sensor, the outgoing images offset from one another in the scanning direction, wherein the outgoing images are projected onto the image sensor such that each pixel of an outgoing image corresponding to a position on the object matches a pixel or group of pixels of another outgoing image corresponding to the respective position on the object; a light generator configured to generate at least two different colours of light, defined by different wavelengths, to illuminate the object; a timing module configured to control the light generator to sequentially generate at least two different colours of light during scanning of a single frame; and a processor configured to identify matched pixels of different outgoing images corresponding to a respective position on the object, and to resolve the colour spectrum of the respective position of the object based on the intensity readout of each matched pixel and the colours of light that the respective matched pixels were exposed to while they were sensitive.

FIELD OF THE INVENTION

The present application relates to an image sensing system fordetermining the colour of an object, and a method for determining thecolour of an object.

BACKGROUND OF THE INVENTION

Multispectral imaging involves making an image with more spectralresolution (i.e. colours) than the three colours that the human eye candistinguish (red, green, and blue). Generally, making an image withcolour involves the use of image sensor which can sense more than onecolour. However, colour sensors are more expensive than black and whiteimage sensors, and sensors which can sense more than 3 colours are yetmore expensive and difficult to make.

US2008/0177185 discloses a skin area detection imaging device fordetecting a skin area of a human body as an object comprises: twooptical lenses to form two unit images on an imaging element bycollecting light from the object illuminated by near-infrared light; arolling shutter for sequentially reading the unit images; and two LEDsfor emitting lights with different wavelengths (850 nm and 940 nm) inthe near-infrared range. A microprocessor switches on the two LEDs whenreading the two unit images, respectively. The skin area of one readunit image is displayed with brightness different from that of the otherunit image based on difference in reflectance to various wavelengths ofnear-infrared light. The microprocessor compares the two unit images todetermine, as a skin area, an area having difference in brightnesslarger than a predetermined value. This makes it possible to detect theskin area in a short time.

SUMMARY OF THE INVENTION

According to a first aspect, there is provided an image sensing systemcomprising: a rolling shutter image sensor comprising an array of pixelsin a frame arranged into a plurality of image lines extending along awidth direction, and distributed in a scanning direction which isperpendicular to the width direction, wherein the rolling shutter imagesensor is configured to scan the frame by sequentially scanning eachimage line along the scanning direction, wherein scanning each imageline includes making the pixels in each image line sensitive to lightfor a predetermined exposure time, and then determining an intensityreadout for each pixel of the respective image line; a lens systemconfigured to project at least two similar outgoing images of an objectonto the image sensor, the outgoing images offset from one another inthe scanning direction, wherein the outgoing images are projected ontothe image sensor such that each pixel of an outgoing image correspondingto a position on the object matches a pixel or group of pixels ofanother outgoing image corresponding to the respective position on theobject; a light generator configured to generate at least two differentcolours of light, defined by different wavelengths, to illuminate theobject; a timing module configured to control the light generator tosequentially generate at least two different colours of light duringscanning of a single frame; and a processor configured to identifymatched pixels of different outgoing images corresponding to arespective position on the object, and to resolve the colour spectrum ofthe respective position of the object based on the intensity readout ofeach matched pixel and the colours of light that the respective matchedpixels were exposed to while they were sensitive.

At least two of the outgoing images may be projected onto the imagesensor in identical size.

The matched pixels may be predetermined based on the lens system, orcould be calculated in real-time by processing the outgoing images andcomparing the pixels from one outgoing image to the pixels of anotheroutgoing image to determined matched pixels.

The lens system may be configured to project an array of outgoing imagesonto the image sensor having at least two distinct columns of outgoingimages. The columns may be distributed along the width direction, eachcolumn comprising at least two outgoing images distributed in thescanning direction, wherein each outgoing image may be offset in thescanning direction from all other outgoing images.

An outgoing image in a column may be split across the frame such that atop portion of one outgoing image is projected on the bottom of theimage sensor and a remaining bottom portion of another outgoing image isprojected on the top of the image sensor in the same column, wherein thetop portion and the bottom portion together define a single wholeoutgoing image.

The exposure time of each image line may be equal to the time taken toread out the number of image lines across which each outgoing imagespans.

The timing module may be configured to control the light generator togenerate different colours of light at intervals, such that the lightgenerator may be controlled to generate at least two different coloursof light during the intensity readout of image lines of at least one ofthe outgoing images in a frame.

The light generator may be configured to sequentially generate thedifferent colours of light in cycles, each cycle may be generated withina frame and may comprise generation of the different colours of light ina predetermined sequence. Each subsequent cycle may have the samesequence as a preceding cycle but begins with the second colour of lightof the preceding cycle and ends with the first colour of light of thepreceding cycle.

The timing module may be configured to control the light generator togenerate light at intervals, such that there are as many intervalsduring the scanning of each outgoing image as there are columns ofoutgoing images projected onto the image sensor.

The light generator may be configured to generate up to as many coloursas there are intervals of light generation in a frame. The intervals maybe regular intervals.

The light generator may be configured to produce flashes of light or toproduce a continuous emission of light.

The processor may be configured, for readout of each and every pixel, toresolve the colour spectrum of the respective position of the objectbased on the intensity readout of the respective pixel and at least onepreceding matched pixel.

According to a second aspect, there is provided a method of resolvingthe colour of an object with a rolling shutter image sensor comprisingan array of pixels in a frame arranged into a plurality of image linesextending along a width direction, and distributed in a scanningdirection which is perpendicular to the width direction, wherein therolling shutter image sensor is configured to scan the frame bysequentially scanning each image line along the scanning direction,wherein scanning each image line includes making the pixels in eachimage line sensitive to light for a predetermined exposure time, andthen determining an intensity readout for each pixel of the respectiveimage line, the method comprising: projecting at least two similaroutgoing images onto the image sensor offset from one another along thescanning direction, wherein each pixel of an outgoing imagecorresponding to a position on the object matches a pixel or a group ofpixels of another outgoing image corresponding to the respectiveposition on the object; sequentially generating at least two differentcolours of light, defined by different wavelengths, to illuminate theobject during scanning of the frame of the image sensor, identifyingmatched pixels of different outgoing images corresponding to arespective position on the objection, and resolving the colour spectrumof the respective position of the object based on the intensity readoutof each matched pixel in the outgoing image and the colours of lightthat the respective matched pixels were exposed to while they weresensitive.

The method may be a computer implemented method.

The method may comprise projecting an array of outgoing images onto theimage sensor in at least two distinct columns of outgoing images. Thecolumns may be distributed along the width direction, each columncomprising at least two outgoing images distributed in the scanningdirection, wherein each of the outgoing images may be offset from allother outgoing images in the scanning direction.

The exposure time may be equal to the time taken to readout the numberof image lines across which each outgoing image spans.

Different colours of light may be generated at intervals, such that atleast two different colours of light are generated during the intensityreadout of image lines of at least one of the outgoing images in aframe.

Different colours of light may be generated sequentially in cycles. Eachcycle may be generated within a frame and may comprise generation of thedifferent colours of light in a predetermined sequence. Each subsequentcycle may have the same sequence as a preceding cycle but begin with asecond colour of light of the preceding cycle and ends with a firstcolour of light of the preceding cycle.

Different colours of light may be generated at intervals, such thatthere are as many intervals during scanning of each outgoing image, asthere are columns of outgoing images projected onto the image sensor.

Generating light may include generating a flash of light or generating acontinuous light.

The projections of the at least two outgoing images on the rollingshutter image sensor may have no overlapping parts (i.e. differentoutgoing images may not occupy the same space on the image sensor).

These and other aspects will be apparent from and elucidated withreference to the embodiment(s) described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments will now be described, by way of example only,with reference to the following drawings, in which:

FIG. 1 schematically shows an example image sensing system;

FIG. 2 schematically shows a roller shutter image sensor with imagesprojected onto it in a first example;

FIG. 3 shows the roller shutter image sensor according to FIG. 2 with ascanning time map;

FIG. 4 schematically shows a roller shutter image sensor with imagesprojected onto it in a second example;

FIGS. 5 a and 5 b show a roller shutter image sensor with imagesprojected onto it in a third and fourth example respectively; and

FIG. 6 is a flow chart showing steps of a method for resolving colour ofan image.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows an image sensing system 10 in use to sense an image of anobject 12.

The image sensing system 10 comprises a rolling shutter image sensor 14and a lens system 16 configured to project three similar outgoing imagesof the object 12 onto the image sensor 14. In some examples, the lenssystem may be configured to project two or more than three similaroutgoing images of the object onto the image sensor.

The lens system 16 in this example comprises three lenses 24 which aredisposed adjacent one another, in front of the image sensor 14. The lenssystem 16 is therefore arranged to be disposed between the object 12 tobe imaged and the image sensor 14.

In other examples, the lens system may comprise a lens and one or morebeam splitters, with the beam splitters disposed between the lens andthe images sensor, such that the beam splitters are configured to splitbeams from the lens to project two or more identical images onto theimage sensor.

Projecting two or more outgoing images onto a single image sensor 14effectively increases the frame rate of the image sensor 14, albeit witha reduced resolution of each image. In this example, since threeoutgoing images are projected onto the image sensor, the frame rate iseffectively tripled compared to normal use of an image sensor. If thereare two images projected onto a single image sensor, the frame rate ofthe image sensor is effectively doubled.

The image sensing system 10 comprises a light generator 18 which, inthis example, is configured to generate three different colours of lightto illuminate the object 12. In this example, the light generator 18comprises three lights 20 which are configured to generate flashes oflight at different wavelengths (i.e. different colours of light). Inother examples, the light generator may be configured to generate twodifferent colours of light with two lights or more than three differentcolours of light. In further examples, the lights may be configured togenerate continuous light, back to back, as opposed to flashes. It willbe appreciated that generating different colours of light can also beachieved with only one light, together with a plurality of differentcolour filters.

FIG. 2 shows the rolling shutter image sensor 14 comprising an array ofpixels 30 in a frame 32 arranged into image lines 34 extending along awidth direction 36, and distributed in a scanning direction 40 which isperpendicular to the width direction 36.

In this example in FIG. 2 , the lens system 16 is configured to projectthree outgoing images 42 onto the image sensor 14, a first outgoingimage 42 a, a second outgoing image 42 b and a third outgoing image 42c. The outgoing images 42 are each offset from one another in thescanning direction 40, such that they do not overlap in the scanningdirection 40. In other words, the three outgoing images 42 are arrangedin a column from the first outgoing image 42 a to the third outgoingimage 42 c in the scanning direction 40. The outgoing images 42 in thisexample are projected in identical size on the image sensor 14 such thateach pixel 30 of one outgoing image 42, corresponding to a position onthe object 12, matches a pixel 30 of another outgoing image 42corresponding to the same position on the object 12. It will beappreciated that the outgoing images need not be of identical size, andthat pixels 30 of one outgoing image corresponding to a position on theobject, may match a group of pixels 30 of another outgoing image,corresponding to the same position on the object.

For example, in FIG. 2 , the first outgoing image 42 a comprises a pixelA, which matches a pixel B in the second outgoing image 42 b, whichmatches a pixel C in the third outgoing image 42 c. Pixels A, B, and Ctherefore correspond to the same position on the object 12 which isbeing imaged.

The rolling shutter image sensor 14 is configured to scan the wholeframe 32 of pixels 30 by scanning each image line 34 sequentially alongthe scanning direction 40. In other words, it is configured to scan animage line 34, followed by the adjacent image line 34 in the scanningdirection 40.

Scanning of each image line 34 includes scanning each pixel 30 in theimage line 34 sequentially along the width direction 36. In other words,the image sensor 14 scans a pixel 30 in an image line 34 followed by anadjacent pixel in the image line 34 in the width direction 36.

Scanning each image line 34 includes making the pixels 30 in therespective image line 34 sensitive to light for a predetermined exposuretime, and then reading out each pixel 30, as is explained in more detailwith reference to FIG. 3 .

FIG. 3 shows the image sensor 14 with the plurality of image lines 34,and the first outgoing image 42 a, the second outgoing image 42 b, andthe third outgoing image 42 c projected onto the image sensor 14. Nextto the image sensor 14, a scanning time map 50 is shown. The scanningtime map 50 shows the passage of time from left to right. For each imageline 34, the scanning time map 50 shows an exposure time 52 followed byan intensity readout 54. In other words, once the image line 34 has beensensitive to light for the predetermined exposure time 52, the imagesensor 14 is configured to readout the pixels 30 in the image line 34.Reading out the pixels 30 comprises determining an intensity for eachpixel 30 of the respective image line 34, corresponding to the intensityof light received at the respective pixel 30 during the exposure time52.

The beginning of the exposure time 52 for each image line 34 in thescanning direction 40 is offset a predetermined amount of time from thepreceding image line 34, corresponding to the amount of time it takes toreadout 54 an image line 34. Therefore, while each image line 34 issensitive to light for the same amount of time, the sensitivity to lightbegins for each image line 34 at a different time.

Flashes of light 56 illuminate the object 12 while only the image lines34 across which a single outgoing image 42 spans are sensitive to light.This is possible due to the staggered sensitivity of each image line 34in the scanning direction 40. As such, the exposure time 52 for eachimage line 34 is equal to the time taken to read out the number of imagelines 34 across which each outgoing image 42 spans. This means that, atany one time, the number of image lines 34 which are sensitive to lightequals the number of image lines 34 across which each outgoing image 42spans.

Each flash of light 56 will therefore only affect the intensity readout54 of the image lines 34 associated with a single outgoing image 42. Inthis example, there are three flashes of light 56, one flash 56corresponding to each outgoing image 42 projected onto the image sensor14.

Referring back to FIGS. 1 and 2 , the image sensing system 10 comprisesa timing module 26 which is configured to control the light generator 18to generate the different colours of light at suitable intervals, inother words to control the different lights 20 of the light generator 18to flash at the correct time.

More specifically, in this example, the timing module 26 is configuredto control the light generator 18 to generate a flash of light 56 a-56 cin one colour at a point when all of the image lines 34 of a singleoutgoing image 42 are sensitive and the image lines 34 of anotheroutgoing image 42 are not sensitive. The timing module 26 is configuredto sequentially generate another flash of light 56 a-c in another colourat a point when all of the image lines 34 of another outgoing image 42are sensitive, and the image lines 34 of other outgoing images 42 arenot sensitive.

In this example, shown in FIGS. 2 and 3 , during the exposure time 52 ofthe first four image lines 34 in the scanning direction 40, across whichthe first outgoing image 42 a spans, the timing module 26 controls thelight generator 18 to generate a flash of red light 56 a.

During the exposure time of the next four image lines 34 in the scanningdirection 40, across which the second outgoing image 42 b spans, thetiming module 26 controls the light generator 18 to generate a flash ofgreen light 56 b.

During exposure time of the last four image lines 34 in the scanningdirection 40, across which the third outgoing image 42 c spans, thetiming module 26 controls the light generator 18 to generate a flash ofblue light 56 c. This is repeated for each scan of the frame 32.

Although the colours red, green and blue have been used in this order inthis example, in other examples, any three colours of light may be used,and the colours of light may be flashed in any order during scanning ofa frame.

The image sensing system 10 further comprises a processor 28 which isconfigured to identify matched pixels 30 of different outgoing images42, such as pixels A, B and C shown in FIG. 2 , which each correspond tothe same position on the object 12.

The processor 28 is configured to resolve the colour spectrum of therespective position on the object 12 based on the intensity readout ofthe matched pixels A, B, and C, and based on which colours of light theobject 12 was exposed to when the respective pixels A, B and C weresensitive, even if the image sensor 14 used is a monochrome imagesensor.

For example, during the exposure (i.e. sensitivity) of pixel A, theobject 12 is illuminated by the flash of red light 56 a. During exposureof pixel B, the object 12 is illuminated by the flash of green light 56b. During exposure of pixel C, the object 12 is illuminated by the flashof blue light 56 c. The intensity readout of pixel A will thereforecorrespond to the amount of red light reflected by the respectiveposition on the object 12, the intensity readout of pixel B willcorrespond to the amount of green light reflected by the respectiveposition on the object 12, and the intensity readout of pixel C willcorrespond to the amount of blue light reflected by the respectiveposition on the object 12.

In this example, the colour X_(c) of the position on the object 12corresponding to pixels A, B and C can be determined by superposing theintensity readouts of the three matched pixels A, B and C.

X _(c) =A+B+C  Equation 1

The processor 28 can therefore resolve the colour X_(c) of therespective position on the object 12 corresponding to matched pixels A,B and C with Equation 1. The resolved colour X_(c) will have a spectralresolution including the wavelengths of the three colours of lightgenerated by the light generator 18 during scanning of the frame 32.

It will be appreciated that this superposition of intensity readouts canbe used with any two or more colours of light, and that the spectralresolution of the resolved colour X_(c) will include the wavelengths ofthe colours of light used to illuminate the object 12 during scanning ofa frame 32.

The superposition of intensity readouts can be achieved with any numberof outgoing images 42 projected onto the image sensor 14, with up to asmany different colours of light flashed as there are outgoing images 42projected onto the image sensor 14. The more colours (i.e. wavelengths)of light there are used to illuminate the object 12 for matched pixels30 during scanning of the frame 32, the higher the spectral resolutionof the resolved colour X_(c) will be.

Sometimes, using fewer colours to illuminate the object 12 for matchedpixels 30 during scanning of the frame 32 can result in a more accuratedetermination of the resolved colour X_(c). For example, if there aretwo or more pixels 30 in a matched group of pixels 30 such as pixels A,B and C which are sensitive during illumination of the object 12 withthe same colour of light, the intensity readout for these pixels 30 canbe used to provide an error measure which the processor 28 can use toresolve the colour X_(c) more accurately.

For example, if during the exposure (i.e. sensitivity) of pixel A, theobject 12 is illuminated by the flash of red light, during exposure ofpixel B, the object 12 is illuminated by the flash of green light, andduring exposure of pixel C, the object 12 is illuminated by anotherflash of red light, then the spectral resolution will include thewavelengths of only the red and the green light, but the intensityreadout of pixel A and C should be the same. If they are not the same,then there is an error in the readout, which can be mitigated by, forexample using the average intensity of pixels A and C, as shown inEquation 2 below and superposing that with the intensity readout ofpixel B.

$\begin{matrix}{X_{c1} = {\frac{A + C}{2} + B}} & {{Equation}2}\end{matrix}$

FIG. 4 shows the roller shutter image sensor 14 with outgoing images 42projected onto it in a different configuration to FIGS. 2 and 3 , whichmakes better use of the whole frame 32 of the image sensor 14.

In this example, the lens system 16 is configured to project an array ofoutgoing images 42 onto the image sensor 14, in this example, nine. Thenine outgoing images 42 are arranged into three columns 60 which aredistributed along the width direction 36. Each column 60 comprises atotal of three outgoing images 42 distributed in the scanning direction40. In other examples, there may be two or more columns of outgoingimages, and there may be two or more outgoing images in each column. Theprojections of the outgoing images 42 on the image sensor 14 have nooverlapping parts (i.e. different outgoing images do not occupy the samespace on the image sensor 14).

Each column 60 of outgoing images 42 is offset from an adjacent column60 of outgoing images 42 in the scanning direction 40, such that in somecolumns, an outgoing image 42 is split across the frame 32 so that aportion of the outgoing image 42 is projected at one end (e.g. the top)of the column 60, and the remaining portion of the outgoing image 42 isprojected at the other end (e.g. the bottom) of the column 60, such thatthe two portions in one column define a single whole outgoing image.

In this example, in a first column 60 a in the width direction 36, eachof the outgoing images 42 is whole. In a second column 60 b in the widthdirection 36, the outgoing images 42 are offset in the scanningdirection 40 by one third of one outgoing image 42 compared to the firstcolumn 60 a. In other words, one third of a portion of an outgoing image42 is projected at the top of the second column 60 b, and the remainingtwo thirds of the respective outgoing image 42 is projected on thebottom of the second column 60 b. In a third column 60 c in the widthdirection 36, the outgoing images are offset in the scanning direction40 by one third of one outgoing image 42 compared to the second column60 b. Therefore, the first column 60 a is also offset by one third ofone outgoing image 42 compared to the third column 60 c.

This effectively increases the frame rate of the image sensor 14nine-fold. The order of reading out each whole outgoing image 42 is thefirst outgoing image in the scanning direction 40 of the first column 60a, followed by the first whole outgoing image 42 in the second column 60b, followed by the first whole outgoing image 42 in the third column 60c, and continues for the second outgoing image 42 of each column 60 andso on.

The timing module 26 controls the light generator 18 to generatedifferent colours of light at regular intervals. In this example, theintervals are spaced such that three flashes 56 of light are generatedduring the exposure of each image line 34. In other words, in the timetaken to readout all of the image lines 34 of one outgoing image 42, thetiming module 26 controls the light generator 18 to generate threeflashes 56 of light. In this example, the number of intervals of flashes56 during exposure of each image line 34 corresponds to the number ofcolumns 60 of outgoing images 42.

In this example, there are three colours of light generated by the lightgenerator 18; red, green and blue (R, G, B). In this example, each flash56 of light is a different colour to the preceding flash 56 of light. Itwill be appreciated that in other examples, there may be two consecutiveflashes of light of the same colour.

The light generator 18 in this example is configured to generate thedifferent colours of light in cycles, where each cycle comprises threeflashes 56 of light of different colours in a predetermined sequence.Each cycle has the same sequence as the preceding cycle but begins withthe second colour of light of the preceding cycle and ends with thefirst light of the preceding cycle.

Therefore, in this example, the first cycle includes generation of lightin the sequence RGB, the second cycle generates light in the sequenceGBR, and the third cycle generate light in the sequence BRG. Each cycleis generated within a single frame, and in this example, there aremultiple cycles generated within a single frame. In other examples, anyother suitable sequence may be used, and the changes between subsequentcycles may be any suitable change.

The processor 28 identifies matched pixels in the nine outgoing images42, corresponding to a single position on an object 12, which isrepresented in this example by pixels A₂-I₂. Each pixel A₂-I₂ issensitive to light during three different flashes 56.

For each pixel, the intensity readout will correspond to the amount oflight reflected by the object 12 from the coloured flashes 56. This canbe represented by Equation 3 below:

A ₂ =R+G+B

B ₂ =G+B+G=2G+B

C ₂ =B+G+B=2B+G

D ₂ =G+B+R

E ₂ =B+R+B=2B+R

F ₂ =R+B+R=2R+B

G ₂ =B+R+G

H ₂ =R+G+R=2R+G

I ₂ =G+R+G=2G+R  Equation 3:

Equations A₂, D₂ and G₂ are the same, which leave seven equations toresolve three unknowns, R, G and B. The processor 28 can resolve theseequations to determine the unknown variables, and thereby to resolve thecolour X_(c) of the position on the object 12 using Equation 1. Onlythree of the equations in Equation 3 are needed to solve the threeunknowns, but if all of the information is used, this can reduce noise.A least square error fit can be used to solve for R, G and B mostaccurately.

This can be repeated for every pixel 30 to resolve the colour X_(c) ofeach position of the object 12 being imaged, even with a monochromeimage sensor. The spectral resolution of the resolved colour X_(c) willbe limited by the wavelengths of the colours of light which are flashed.Therefore, the more colours of light which are flashed, the higher thespectral resolution will be for the resolved colour X_(c). There can beas many colours of light flashed as there are outgoing images 42.Therefore, with nine outgoing images 42, there could be nine differentcolours of light flashed, which would give 9 different equations for thematched pixels to resolve the intensities of the nine different colours.

Although it has been described that there are three columns of outgoingimages projected onto the image sensor, it will be appreciated thatthere may be any suitable number of columns, n, of images distributedalong the width direction, and that the columns may be offset fromadjacent columns by 1/n of an outgoing image. This ensures that each andevery outgoing image 42 which is projected onto the image sensor will beoffset from each and every other outgoing image 42, such that no twooutgoing images 42 will be exposed to light at identical times.

The resolved colour X_(c) of the object 12 can therefore be determinedafter readout of each frame 32, using matched pixels for the whole frame32, such that the colour resolution has a frame rate equal to the framerate of the image sensor 14. However, the resolved colour X_(c) of theobject 12 could also be determined after readout of each matched pixel30, if the readout of the previous eight matched pixels are used on arolling basis. For example, when the intensity readout of pixel E₂ iscarried out, the intensity readout of pixel E₂, together with theintensity readout of pixel A₂-D₂ of the same frame 32, and the intensityreadouts of pixels F₂-I₂ of the previous frame 32 can be used to resolvethe colour X_(c) of the object 12 at the position corresponding to pixelE₂. This increases the effective frame rate of the colour resolution bythe number of outgoing images 42 projected onto the image sensor 14, inthis case, nine-fold.

In an example in which the lights are configured to generate continuouslight (i.e. continuous illumination), back-to-back, rather than flashesof light, the equation for intensity for each pixel will differslightly. For example, for a pixel Z, the intensity of light receivedwhile the pixel is sensitive will be based on the proportion of timethat the pixel was exposed to each colour of light while it wassensitive, such as:

Z=t ₁ R+t ₂ G+t ₃ B

where t ₁ +t ₂ +t ₃=1.

In a similar manner to the example described above, each matched pixelwould have a corresponding equation, and provided there are as manydifferent equations for the matched pixels as there are differentcolours of light, the processor can resolve the colour of the positionof the object to which the matched pixel corresponds with theseequations.

Having multiple outgoing images 42 projected onto the image frame, andmultiple intervals of light generation during scanning of a single frame32, means that there is a higher frequency of light change than, forexample, having only one interval of colour light generation duringscanning of the frame. Having such high frequency light colour changesmeans that the light changes become less visible to the user, eventuallyperceived by the user only as a continuous light, rather than rapidflashing lights.

FIGS. 5 a and 5 b show third and fourth examples respectively ofprojections of outgoing images 42 onto an image sensor 14.

In FIG. 5 a , there are three outgoing images 42 projected in a column160 in a similar manner to the first column 60 a of FIG. 4 , and onelarger outgoing image 142 projected onto the remaining space of theimage sensor 14. Such an arrangement may be useful if it is desirable tohave a high frame rate at a lower spatial resolution for some features,and a higher spatial resolution at a lower frame rate for others. Thelarger outgoing image 142 would therefore provide a higher spatialresolution than the column 160 of outgoing images 42, but in order todetermine the colour of each pixel, the frame would have to be scannedseveral times, and it would have to be ensured that each pixel of thelarger outgoing image 142 was exposed to different colours of light ineach subsequent scan of the frame 32.

In FIG. 5 b , there are two outgoing images 242 projected in a column260 onto the image sensor 14, where the projected images have beenstretched in the width direction 36 to cover as much of the image sensor14 as possible. This increases the horizontal resolution of the outgoingimages 242.

FIG. 6 is a flow chart showing a method 300 of resolving the colour ofthe object 12 with the rolling shutter image sensor 14. The method 300starts with block 302 to project at least two outgoing images 42, 142,242 onto the image sensor 14 offset from one another along the scanningdirection, such as shown in FIGS. 2-5 . Within the outgoing images 42,142, 242 each pixel 30 of the outgoing images 42, 142, 242 matchesanother pixel 30 or group of pixels of another outgoing image 42, 142,242 and corresponds to a position on the object 12.

In block 304, the method 300 includes sequentially generating at leasttwo different colours of light having different wavelengths, toilluminate the object 12. The light generation may be controlled to begenerated in intervals, for example as described with reference to FIGS.2 and 3 , and FIG. 4 .

In block 306, the method 300 includes scanning pixels 30 of the imagesensor 14 to obtain an intensity readout for each pixel 30. In thisexample, the exposure time of each image line 34 of the image sensor 14is equal to the time taken to readout the number of image lines 34across which each of the outgoing images 42 spans.

In block 308, the method 300 includes identifying matched pixels ofdifferent outgoing images 42, 142, 242, corresponding to a position onthe object 12, and block 310 includes resolving the colour spectrumX_(c) of the position on the object 12 based on the intensity readout ofeach matched pixel 30, and the colours of light that the respectivepixels 30 were exposed to while they were sensitive to light.

It will be appreciated that all of the blocks 302-308 can be carried outat the same time.

Although it has been described that the timing module controls the lightgenerator to generate different colours of light at regular intervals,the intervals need not be regular, as a processor could resolve thecolours with irregular intervals of light generation as well.

Further, it has been described that the processor identifies matchedpixels in different outgoing images. The matched pixels may bepredetermined based on the lens system, or could be calculated inreal-time by processing the outgoing images and comparing the pixelsfrom one outgoing image to the pixels of another outgoing image. It willbe apparent that the accuracy of the colour resolution may be impactedby motion of the object during scanning. The less motion during scanningthe better the colour resolution of each point on the object. Using aprocessor to compare pixels in real-time to determine matched pixels mayallow for some compensation of motion during scanning.

Variations to the disclosed embodiments can be understood and effectedby those skilled in the art in practicing the principles and techniquesdescribed herein, from a study of the drawings, the disclosure and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor or other unit may fulfil thefunctions of several items recited in the claims. The mere fact thatcertain measures are recited in mutually different dependent claims doesnot indicate that a combination of these measures cannot be used toadvantage. A computer program may be stored or distributed on a suitablemedium, such as an optical storage medium or a solid-state mediumsupplied together with or as part of other hardware, but may also bedistributed in other forms, such as via the Internet or other wired orwireless telecommunication systems. Any reference signs in the claimsshould not be construed as limiting the scope.

1. An image sensing system comprising: a rolling shutter image sensorcomprising an array of pixels in a frame arranged into a plurality ofimage lines extending along a width direction, and distributed in ascanning direction which is perpendicular to the width direction,wherein the rolling shutter image sensor is configured to scan the frameby sequentially scanning each image line along the scanning direction,wherein scanning each image line includes making the pixels in eachimage line sensitive to light for a predetermined exposure time, andthen determining an intensity readout for each pixel of the respectiveimage line; a lens system configured to project an array of similaroutgoing images of an object onto the image sensor, the array comprisingat least two distinct columns of outgoing images, wherein the columnsare distributed along the width direction, each column comprising atleast two outgoing images distributed in the scanning direction, each ofthe outgoing images offset from all other outgoing images in thescanning direction, wherein the outgoing images are projected onto theimage sensor such that each pixel of an outgoing image corresponding toa position on the object matches a pixel or group of pixels of anotheroutgoing image corresponding to the respective position on the object; alight generator configured to generate at least two different colours oflight, defined by different wavelengths, to illuminate the object; atiming module configured to control the light generator to sequentiallygenerate at least two different colours of light during scanning of asingle frame; and a processor configured to identify matched pixels ofdifferent outgoing images corresponding to a respective position on theobject, and to resolve the colour of the respective position of theobject based on the intensity readout of each matched pixel and thecolours of light that the respective matched pixels were exposed towhile they were sensitive.
 2. An image sensing system according to claim1, wherein the exposure time of each image line is equal to the timetaken to read out the number of image lines across which each outgoingimage spans.
 3. An image sensing system according to claim 1, whereinthe timing module is configured to control the light generator togenerate different colours of light at intervals, such that the lightgenerator is controlled to generate at least two different colours oflight during the intensity readout of image lines of at least one of theoutgoing images in a frame.
 4. An image sensing system according toclaim 3, wherein the light generator is configured to sequentiallygenerate the different colours of light in cycles, each cycle beinggenerated within a frame and comprising generation of the differentcolours of light in a predetermined sequence, wherein each subsequentcycle has the same sequence as a preceding cycle but begins with thesecond colour of light of the preceding cycle and ends with the firstcolour of light of the preceding cycle.
 5. An image sensing systemaccording to claim 1, wherein the timing module is configured to controlthe light generator to generate light at intervals, such that there areas many intervals during the scanning of each outgoing image as thereare columns of outgoing images projected onto the image sensor.
 6. Animage sensing system according to claim 1, wherein the light generatoris configured to produce flashes of light or to produce a continuousemission of light.
 7. An image sensing system according to claim 1,wherein the processor is configured, for readout of each and everypixel, to resolve the colour spectrum of the respective position of theobject based on the intensity readout of the respective pixel and atleast one preceding matched pixel.
 8. A method of resolving the colourof an object with a rolling shutter image sensor comprising an array ofpixels in a frame arranged into a plurality of image lines extendingalong a width direction, and distributed in a scanning direction whichis perpendicular to the width direction, wherein the rolling shutterimage sensor is configured to scan the frame by sequentially scanningeach image line along the scanning direction, wherein scanning eachimage line includes making the pixels in each image line sensitive tolight for a predetermined exposure time, and then determining anintensity readout for each pixel of the respective image line, themethod comprising: projecting an array of similar outgoing images ontothe image sensor in at least two distinct columns of outgoing images,wherein the columns are distributed along the width direction, eachcolumn comprising at least two outgoing images distributed in thescanning direction, wherein the outgoing images are each offset from allother outgoing images along the scanning direction, wherein each pixelof an outgoing image corresponding to a position on the object matches apixel or a group of pixels of another outgoing image corresponding tothe respective position on the object; sequentially generating at leasttwo different colours of light, defined by different wavelengths, toilluminate the object during scanning of the frame of the image sensor;identifying matched pixels of different outgoing images corresponding toa respective position on the objection, and resolving the colour of therespective position of the object based on the intensity readout of eachmatched pixel in the outgoing image and the colours of light that therespective matched pixels were exposed to while they were sensitive. 9.A method according to claim 8, wherein the exposure time is equal to thetime taken to readout the number of image lines across which eachoutgoing image spans.
 10. A method according to claim 8, whereindifferent colours of light are generated at intervals, such that atleast two different colours of light are generated during the intensityreadout of image lines of at least one of the outgoing images in aframe.
 11. A method according to claim 10, wherein different colours oflight are generated sequentially in cycles, each cycle being generatedwithin a frame and comprising generation of the different colours oflight in a predetermined sequence, wherein each subsequent cycle has thesame sequence as a preceding cycle but begins with a second colour oflight of the preceding cycle and ends with a first colour of light ofthe preceding cycle.
 12. A method according to claim 11, whereindifferent colours of light are generated at intervals, such that thereare as many intervals during scanning of each outgoing image, as thereare columns of outgoing images projected onto the image sensor.
 13. Amethod according to claim 8, wherein generating light includesgenerating a flash of light, or generating a continuous light.